Video Interviewing: A Best Practices Discussion

There is a rich body of literature in I-O Psychology that helps outline best practices when creating selection measures in organizations (e.g., Campion et al., 1997; Huffcutt & Arthur, 1994; Morgeson & Campion, 1997; Sackett & Lievens, 2008). Some of these best practices include structured and standardized approaches in the way questions are asked and scored. These guidelines are, to some degree, easier to follow when interviews occur in person because the environment can be heavily controlled by the organization. However, in the current climate, more organizations are opting in to utilizing video interviews which should not be regarded as comparable to in person interviews. The current panel discussion addressed the research and practice gap that exists with more novel methods. This panel was discussed at the 2021 SIOP conference.

Yuliya Cheban, Ethan Ray, and Charles Scherbaum https://ethanray11.github.io/ (Baruch College)https://www.iopsych-baruch.com
04-26-2021

Recently, technology has advanced the ways in which organizations can conduct selection interviews. A 2015 survey performed by Futurestep, a Korn Ferry company, found that 71% of the seven-hundred executives they surveyed use video-interviewing at their organization (Futurestep, 2015). Given the increase in technology-integrated workplaces over the past decade and the rapid move to remotely delivered selection processes due to the COVID-19 pandemic, we can likely assume that these numbers have greatly increased. There are a number of video interview benefits that have fueled this increased adoption, such as the reduced costs in the form of travel expenses and time it takes to individually interview applicants, increased flexibility for both applicants and organizations, expanded applicant pools, increased efficiency, and interview standardization (Blacksmith et al., 2016; Brenner et al., 2016; Langer et al., 2017). With over 100 online vendors offering access to their video interview solutions (Software Advice, 2020) organizations are faced with the decision to conduct either two-way or one-way video interviews.

The difference between one-way and two-way interviews exists in the synchronicity of the interview experience. Two-way video interviews, also known as synchronous video interviews, are a form of communication in which applicants and evaluators communicate via a video conference (Straus et al., 2001). Applicants and evaluators see and hear one another in real-time through a videoconference platform (Langer et al., 2017). One-way interviews, also known as asynchronous video interviews, allow applicants to record their answers to interview questions and submit them via an online platform (Gorman et al., 2018). A one-way video interview process typically involves a web link that is sent to prospective applicants inviting them to an online platform to complete the interview (Torres & Gregory, 2018). Once the invitation is accepted, applicants are prompted to record their responses to predetermined interview questions on camera and submit those responses via the online platform (Brenner et al., 2016). The recorded responses are reviewed later at the convenience of an evaluator, be that a hiring manager, recruiter, or artificial intelligence (Woods et al., 2019). Although interchangeable as current methods of applicant interviews between organizations, there are remarkable differences between two-way and one-way interviews that can influence both the candidates’ experience and the usefulness of the interview.

Establishing Best Practices

First, several differences between in-person and video interviews need to be addressed when considering best practices for video interview procedures. The two most critical factors to consider when making the comparisons among existing methods are fairness and information. Fairness considerations include: (a) determining whether interviewees are each getting the same amount of time to answer questions, or whether time accommodations are given to specific people that need extra time, (b) whether interviews are viewed in their entirety by evaluators, and (c) how scoring is conducted (e.g., human or AI). In contrast, information considerations focus on how information is lost and gained, and include: (a) what do we learn about an applicant, or organization, when we come face-to-face with them versus when we see them on video, (b) what information is gained from an applicant conducting a video interview from their home environment, and (c) what information is lost by restricting interview communication to video or recorded responses? Below, we briefly review video interviews in comparison to face-to-face interviews as well as provide a comparison of both video interview formats on these dimensions.

Video Interviews vs. Face-to-Face

Traditionally, interviews are thought of as a face-to-face interaction between an interviewer and interviewee (Campion et al., 1997). Thus, it is valuable to discuss fairness and available information during video interviews as they compare to face-to-face interviews because face-to-face interviews are used as an anchor. When we compare fairness between these types of interviews, it is important to consider the extent to which they adhere to the 15 core components of good structured interviews (Campion et al., 1997), which include: (1) time spent interviewing, (2) time spent scoring, (3) types of scoring procedures and how they may differ, and (4) differences in impression management potential. In addition, from an information perspective, we can think about what is lost and gained in both settings. It is important to note that the information that is gained in a video interview setting may be non-KSAO related (e.g., posters on an interviewee’s wall; dogs barking in background), which links obtaining information directly to considerations around fairness. Ultimately, the information lost in video interviews may be in the form of lower media richness in video interviews (e.g., non-verbal behaviors, eye-contact). However, increased structure in video interviews may lead to greater job relevant information gained from the interview experience.

One-Way vs. Two-Way Video Interviews

Using this same fairness-information framework, we can compare one-way and two-way video interviews. From a fairness perspective, we can consider: (1) time allotted for interviewing/answering questions, (2) time spent reviewing and scoring (3) applicant reactions to both mediums, (4) influences on organizational attraction, (5) influences on evaluator ratings, and (6) applicant performance between mediums. From an information perspective, we can consider: (1) information gained from follow-up questions and interactivity in two-way interviews, (2) information lost from a lack of interpersonal communication available in one-way interviews, and (3) information gained in the form of increased structure one-way interviews.

Both of the dimensions discussed are critical to consider from applicant and evaluator perspectives. Thoughtfully considering the dimensions when creating selection assessments from both of these lenses can lead to a more well-rounded approach to organizational decision making in their selection process. The chair and co-chairs of the session have carefully collected the panelists to discuss some of these issues.

Chair/Co-Chair Biographies

Dr. Charles A. Scherbaum is a professor of I-O psychology at Baruch College, City University of New York. His research focuses on talent management, cognitive ability testing, test validation, and eye tracking. Publications of his research have appeared in a variety of top tier outlets. He has authored a book on analyzing quantitative data published by Sage. Dr. Scherbaum has been awarded the 2011 M. Scott Myers Award for Applied Research and the 2018 Adverse Impact Reduction Research Initiative and Action Award from the Society for Industrial and Organizational Psychology as well as the 2011 and 2017 Innovations in Assessment Award from the International Personnel Assessment Council for his research on cognitive ability tests and test validation. Dr. Scherbaum has consulted with a variety of private and public organizations. He has worked as an expert in employment ligation in both the private and public sectors including as an expert for the U.S. Department of Justice and the EEOC. Dr. Scherbaum teaches courses on analytics, performance management, and psychometrics in the U.S., Taiwan, and Singapore.

Yuliya Cheban is a doctoral student at The Graduate Center, City University of New York. Broadly, her research focuses on applicant reactions and assessments. In conducting her research, she aims to use a host of methodologies including eye-tracking, think-alouds, and surveys. Yuliya also teaches I-O psychology courses across the City University of New York system at the undergraduate and master’s level. Yuliya graduated with her master’s degree in I-O psychology from California State University, Long Beach. Outside of academia, she has previously worked in the public safety selection division for the City of Los Angeles. Currently, Yuliya works on several assessment consulting projects for public and private organizations.

Ethan Ray is a doctoral student at The Graduate Center, City University of New York. His primary research interests focus on technology-mediated interviews and evaluator ratings. Additionally, he is involved in multiple ongoing research efforts addressing a broad variety of topics including leadership behaviors, employee incivility, and interview best practices. Ethan also teaches I-O psychology courses across the City University of New York system at the undergraduate level. Currently, he works on consulting projects for private organizations.

Panelist Biographies

Dr. Julie McCarthy is Professor of Organizational Behavior and Human Resource Management in the Department of Management at the University of Toronto Scarborough. Julie’s research interests include the study of how organizations can ensure that their policies and practices are viewed favorably by job applicants and employees. She has played multiple roles for SIOP, including an executive member of the SIOP the Science-Practice Task force, and former Chair of the SIOP/Alliance Program Committee. Julie also serves on multiple editorial boards, including the Journal of Applied Psychology and Personnel Psychology, and her research is generously funded by SSHRC. In the corporate sector, Julie has worked with a number of private and public sector corporations in their development of personnel selection tools, including structured interviews.

Dr. Lynn A. McFarland is an Assistant Professor in the Department of Management at the University of South Carolina. Her primary research interests include recruitment, staffing, and workplace diversity. She has published in several leading psychology and management journals such as the Journal of Applied Psychology, Personnel Psychology, and Journal of Management, and has presented over 65 papers at national conferences. She is also the President and co-founder of Human Capital Solutions, Inc., an HR consulting firm founded in 2004 specializing in staffing and performance management.

Dr. Lilia Hayrapetyan is the Global Head of People Analytics and Insights at Citi. Prior to this role, Lilia led firm-wide organizational assessments and served as the Senior Vice President of Org Development and HR Strategy. Before joining Citi, Lilia led the talent management, succession planning, performance management and engagement practices at XL Catlin. Lilia’s earlier experiences include talent management, leadership development and organizational research at Pfizer and PepsiCo and consulting with Mercer and Oliver Wyman, where she advised organizations on engagement surveys, conducted cultural assessments of organizations in preparation of mergers, and developed assessments. Lilia received her Ph.D. from the Graduate Center at the City University of New York.

Dr. Daly Vaughn serves as an Account Director at Modern Hire. Daly is a trusted advisor to some of the world’s leading private sector organizations. He specializes in guiding the development, implementation, and ongoing sustainment of full candidate journey hiring experiences, including text screening, asynchronous and live digital interviews, and multi-method pre-hire selection tools available across all device modalities. He has guided best hiring practices to create innovative staffing solutions across diverse industries such as banking, retail, healthcare, and manufacturing, and has led numerous consulting engagements to design job-relevant game-like simulations, explore innovative item types and approaches, and deliver and evolve mobile-enabled tools. In addition to practical experience in employee selection, Daly conducts research on topics related to social media use in work contexts. Daly received his PhD in Industrial-Organizational Psychology from Auburn University.

Vivian Chou is a Global Talent Assessment Partner at Facebook. She has previously worked as a consultant for BTS where one of her responsibilities was to develop virtual selection and development assessments. Her research interests are focused on the use of neuroscience techniques, diversity and adverse impact issues, and cognitive ability testing as a selection procedure. Her thesis and dissertation research utilize an eye tracker to collect cognitive pupillometry, or fluctuations in pupil size that indicate changes in cognitive resource usage and information processing level. Her master’s thesis research received the 2018 Adverse Impact Reduction Research Initiative (AIRRIA) research grant from the Society for Industrial and Organizational Psychology (SIOP). Vivian received her master’s degree in I/O Psychology at Baruch College.

Dr. Drew Lam is a Talent Management Manager at PepsiCo, where he is responsible for leading frontline selection for the Frito-Lay division, which hires over 15,000 candidates annually. In his role, Drew leads all prescreen and assessment work, oversees the background and drug screen process, and manages the recruitment process outsourcing (RPO). Previously he was an Organization Development Senior Manager at Shearer’s Snacks, and a Talent Management Principal Analyst at The Timken Company and TimkenSteel. Drew is the 2018 recipient of the Crain’s Cleveland Business Archer Award for Organizational Development. Drew received his Ph.D. in I/O Psychology at The University of Akron.

Questions for Panelists The session was designed around the following questions, constructed based on the framework presented above, and posed to the panelists by the chairs.

General

  1. What trends are you seeing in either the literature or in the field when it comes to video interviews?
  2. What do you see as the major research needs currently?
  3. From a test developer view, what are your insights on the differences between face-to-face versus video interviews?
  1. What about the differences between one-way and two-way interviews?
  2. Potential follow-up: What are the most challenging questions that clients are asking about these interviews?
  3. Potential follow-up: What about from a researcher’s perspective?
  4. Potential follow-up: What about from an evaluator’s perspective?
  5. Potential follow-up: What about from an applicant’s perspective?

Fairness

  1. In terms of designing, scoring, and communicating with applicants, what do you see as best practice when delivering a one-way and two-way interview?
  1. Potential follow-up: In asynchronous interviews, if questions are not provided beforehand, should there be preparation time allotted before interviewees are to record their response?
  2. Potential follow-up: What are potential problems in scoring video interviews (i.e., biases) that do not typically emerge in an in-person setting?
  3. Potential follow-up: In early 2020, Illinois mandated that organizations provide applicants with information about AI if it is used to score their recorded interview. What implications do you see this has for an organization?
  4. Potential follow-up: Some video interview platforms talk about a strength of their product being ability to review many more applicants in a shorter time span. Could this have any negative effects on evaluator perceptions? Information
  1. Much of the advice on video interviews for interviewees is centered around aspects of their environment they may not be able to control (e.g., stable internet connection). Considering the inequities surrounding those impacted by these uncontrollable factors, what more practical advice do you have for organizations and applicants?
  1. Potential follow-up: Do you see non-job relevant information gained from video interviews as having influences on applicant ratings?
  1. Although these technologies are increasingly becoming adopted, are face-to-face interviews still the best modality to gain information about an applicant?
  1. Potential follow-up: What practices would you suggest to boost the information gained about applicants during video interviews?